BAB^-CONF-05/018 
SLAC-PUB-11377 
July 2005 



Measurement of 7 in D^*''K^ and DK*^ decays with a 



Dalitz analysis of D ^ K'^tt it' 



The BABAR Collaboration 
February 7, 2008 



Abstract 

We present a measurement of the Cabibbo-Kobayashi-Maskawa CP- violating phase 7 with a Dalitz 
plot analysis of neutral D-meson decays to the Kgir^ir'^ final state from 

BT ^ L>(*)k=f and 

DK*^ decays, using a sample of 227 million BB pairs collected by the BABAR detector. We 
measure 7 = (67 it 28 it 13 it 11)°, where the first error is statistical, the second is the experimental 
systematic uncertainty and the third reflects the Dalitz model uncertainty. This result suffers from 
a two-fold ambiguity. The contribution to the Dalitz model uncertainty due to the description of 
the TTTT S-wave in Kg7r~7r~^, evaluated using a K-matrix formalism, is found to be 3°. 
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1 INTRODUCTION 



CP violation in the Standard Model is described by a single phase in the Cabibbo-Kobayashi- 
Maskawa (CKM) quark-mixing matrix [1]. Although CP violation in the B system is now well 
established, further measurements of CP violation are needed to overconstrain the Unitarity Tri- 
angle [2] and confirm the CKM model or observe deviations from its predictions. The angle 7 of 
the Unitarity Triangle is defined as 7 = arg [— ]. Various methods [3, 4] have been 
proposed to extract 7 using D^K^ ^ decays, all exploiting the interference between the 

color allowed B^ D^K^ (oc Vch) and the color suppressed B^ D^K~ (oc Vub) transitions, 
when the D° and are reconstructed in a common final state. The symbol indicates either a 

or a meson. The extraction of 7 with these decays is theoretically clean because the main 
contributions to the amplitudes come from tree-level transitions. 

Among the decay modes studied so far the Kg7r~7r"*' channel is the one with the highest 
sensitivity to 7 because of the best overall combination of branching ratio magnitude, — D° 
interference and background level. Both BABAR [5] and Belle [6] have reported on a measurement of 
7 based on B^ D^*^'^K^ decays with a Dalitz analysis of K'^tt^tt'^ . Here, the symbol "(*)" 

refers to either a D or D* meson. Belle has recently shown a preliminary result using B~ D^K*~ , 
K*~ — ifgTr" [7] . In this paper we report on the update of the 7 measurement with the addition 
of the B- D^K*-, K*' K^tt' decay mode to the previously used B D^*^^K channels. 

Assuming no CP asymmetry in D decays and neglecting the B^ (Kg'n'^)-aon-K* contri- 

bution, the S=F ^ b^K*^, K°7r-7r+, K*^ K^tt"^ decay chain rate T^{m'i,mX) can be 

written as 

T^{ml,ml) (x\Adt\'^ + rl\AD±\'^ + 2{x^Re[ADT^h±]+yTM-^DT-^h±]} . (1) 

where m?. and are the squared invariant masses of the Kg7r~ and i^svr"*" combinations re- 
spectively from the decay, and Ad^ = AD{'m'^,m'^), with Ad- {Ad+) the amplitude of the 
D*^ Kg7r~-K~^ {D^ — > KgTr~^'K~) decay. In Eq. (1) we have introduced the Cartesian coordinates 

= rBCOs{5B T 7) and y^i = rBsm{SB T l) [5], for which the constraint = x"^ + holds. 
Here, is the magnitude of the ratio of the amplitudes A{B~ — D^K*~) and AiB" —>■ D^K*~) 
and <5b is their relative strong phase. 

In the case where a B^ — > {K^TT^)-n.on-K* component interferes with B^ D^K*^, we 
write a general parameterization of the decay rates following the approach proposed in Ref. [8], 

T^{ml,ml)(x\AD^\'^ + r^s\^D±\'^ + 2[xs^Re[ADT^hJ+ysT'^^[AD^Ah±]} , (2) 

where Xg^: = KrsCOs{Ss T 7)) Z/sq: = «;r5sin((5s ^ 7) and x^-p -|- y^^ = /c^r^, with < /c < 1. In 
the limit of a null B^ D^{Kg7r^)iion-K* contribution, k — > 1, ^ and Sg — > 6b- The 
parameterization given by Eq. (2) is also valid in the case when and 6b happen to vary within 
the K* mass window, and accounts for efficiency variations as a function of the kinematics of the 
B decay. 

Once the decay amplitude Ad is known, the Dalitz plot distributions for from B~ and 
5+ decays can be simultaneously fitted to r_(m^,m^) and T^(m?_,m'^) as given by Eq. (2), 
respectively. A maximum likelihood technique can be used to estimate the CP- violating parameters 
Xgzf, ys^-i and r^. Since the parameter is also floated our {xs^:,ys^) results do not depend on 
any assumption on the amount and nature of the B~ -D°(K°7r^)non-ii"* component, while on 

''Reference to the charge-conjugate state is implied here and throughout the text unless otherwise specified. 
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average the statistical uncertainties do not increase. Moreover, this general treatment allows us to 
consider the B~ — D^{Kg'K^)uon-K* events like B~ — >■ D^K*~ signal. 

Since the measurement of 7 arises from the interference term in Eq. (2), the uncertainty in 
the knowledge of the complex form of can lead to a systematic uncertainty. Two different 
models describing the — > K^'k~'k~^ decay have been used in this analysis. The first model (also 
referred to as Breit-Wigner model) [9] is the same as used for our previously reported measurement 
of 7 on B~ —>■ D^*^^K~, —>■ K°7r~7r+ decays [5], and expresses Ad as a sum of two-body 
decay-matrix elements and a non-resonant contribution. In the second model (hereafter referred 
to as the tttt S-wave K-matrix model) the treatment of the tttt S-wave states in — > K^n~n~^ 
uses a K-matrix formalism [10, 11] to account for the non-trivial dynamics due to the presence 
of broad and overlapping resonances. The two models have been obtained using a high statistics 
flavor tagged sample (D*+ -D'^7r+) selected from e+e~ cc events recorded by BABAR. 

2 THE BABAR DETECTOR AND DATASET 

The analysis is based on a sample of 227 million BB pairs collected by the BABAR detector at 
the SLAC PEP-II e"*"e~ asymmetric-energy storage ring. BABAR is a solenoidal detector optimized 
for the asymmetric-energy beams at PEP-II and is described in [12]. We summarize briefly the 
components that are crucial to this analysis. Charged-particle tracking is provided by a five- 
layer silicon vertex tracker (SVT) and a 40-layer drift chamber (DCH). In addition to providing 
precise spatial hits for tracking, the SVT and DCH also measure the specific ionization (dE/dx), 
which is used for particle identification of low-momentum charged particles. At higher momenta 
{p > 0.7 GeV/c) pions and kaons are identified by Cherenkov radiation detected in a ring-imaging 
device (DIRC). The typical separation between pions and kaons varies from 8a at 2 GeV/c to 
2.5(7 at 4 GeV/c. The position and energy of neutral clusters (photons) are measured with an 
electromagnetic calorimeter (EMC) consisting of 6580 thallium-doped Csl crystals. These systems 
are mounted inside a 1.5-T solenoidal super-conducting magnet. 

3 EVENT SELECTION 

We reconstruct the decays B" D^K*~ with K'^tt-tt^, K*' K^tt- and K° 7r-7r+. 

The Kg candidates are formed from oppositely charged pions with a reconstructed invariant mass 
within 9 MeV/ of the nominal Kg mass [2] . The two pions are constrained to originate from the 
same point. The candidates are selected by combining mass constrained K^ candidates with two 
oppositely charged pions having an invariant mass within 12 McV/c^ of the nominal mass [2]. 
The candidates are mass and vertex constrained. The K*~ candidates are selected from com- 
binations of a Kg with a negative charged pion with an invariant mass within 55 MeV/c^ of the 
nominal K*~ mass [2]. The cosine of the angle between the direction transverse to the beam con- 
necting the or K*^ and the Kg decay points (transverse flight direction), and the Kg transverse 
momentum vector is required to be larger than 0.99. Since the K*~ in B~ — > D^K*~ is polarized, 
we require \ cos 9h\ > 0.35, where Oh is the angle in the K*~ rest frame between the daughter 
pion and the parent B momentum. The distribution of cos 9h is proportional to cos^ 6h for the 
B~ D^K*~ signal and is roughly flat for the e~^e~ qq {q = u, d, s, c) continuum background. 
The B~ candidates are reconstructed by combining a candidate with a K*~ candidate. We select 
the B mesons by using the beam-energy substituted mass, mss = y {^^"^ /2 + pi • y>-bY / — p'^, 
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and the energy difference AE = — E*/2, where the subscripts i and B refer to the initial 
e~^e~ system and the B candidate, respectively, and the asterisk denotes the center-of-mass (CM) 
frame. The resolutions of mES ^-nd AE, evaluated on simulated signal events, are 2.6 MeV/c^ and 
11 MeV, respectively. We define a selection region through the requirement \AE\ < 25 MeV and 
mES > 5.2 GeV/c^. To suppress the background from continuum events we require | cos^tI < 0.8 
where Ox is defined as the angle between the thrust axis of the B candidate and that of the rest of 
the event. After all the cuts are applied the average number of candidates per event is 1.06. We 
select one B candidate per event by taking the one that has the minimum value of a built with 
the and K* masses, resolutions and intrinsic width for the case of the K*. The B~ — > D^K*~ 
reconstruction efficiency is 11.1 ±0.5% for simulated events. The reconstruction purity in the signal 
region mEs > 5.272 GeV/c^ is estimated to be 46%. Figure 1 shows the rriES distribution after all 
the selection criteria are applied. 




(GeV/c^) 



Figure 1: B — > D^K* tues distribution after all selection criteria are applied. The curves 
represent the fit projections for signal plus background (solid) and background (dotted). 



4 THE K^^7T-7T+ DECAY MODEL 

The K^TT~TT^ decay amplitude ^_d("i?., m^) is determined from an unbinned maximum- 

likelihood Dalitz fit to the Dalitz plot distribution of a high-purity (97%) sample of 81496 D*^ — s- 
D^TT^ decays reconstructed in 91.5 fb~^ of data, shown in Fig. 2(a). We use two different models 
to describe ^_D(m^,m^). 

The first (Breit-Wigner) model is the same as used for our previously reported measurement of 
7 on IT vr"*" decays [5]. Here, the decay amplitude is expressed as a sum 

of two-body decay-matrix elements (subscript r) and a non-resonant (subscript NR) contribution. 
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where each term is parameterized with an amphtude and a phase The function Ar{m'^,m'^) 
is the Lorentz-invariant expression for the matrix element of a meson decaying into Kg7T~Tr~^ 
through an intermediate resonance r, parameterized as a function of position in the Dahtz plane. 
For r = p(770) and /9(1450) we use the functional form suggested in Ref. [13], while the remaining 
resonances are parameterized by a spin-dependent relativistic Breit-Wigner distribution [2]. The 
model consists of 13 resonances leading to 16 two-body decay amplitudes and phases (see Table I 
in Ref. [5]), plus the non-resonant contribution, and accounts for efficiency variations across the 
Dalitz plane and the small background contribution (with uniform Dalitz shape). All the resonances 
considered in this model are well established except for the two scalar tttt resonances, a and a', 
whose masses and widths are obtained from our sample. The a and a' resonances are introduced 
in order to obtain a better fit to the data, but we consider in the evaluation of the systematic 
errors the possibility that they do not actually exist. Wc estimate the goodness of fit through a 
two-dimensional test and obtain = 3824 for 3054 — 32 degrees of freedom. This model is the 
one used as nominal in this analysis. 

The second (tttt S-wave K-matrix) model uses the K-matrix formalism [10, 11] to parameterize 
the S-wavc component of the vrvr system in K'^7r~7r~^. The K-matrix approach can be applied 

to the case of resonance production in multi-body decays when the two-body system in the final 
state is isolated, and the two particles do not interact simultaneously with the rest of the final state 
in the production process. In addition, it provides a direct way of imposing the unitarity constraint 
that is not guaranteed in the case of the Breit-Wigner model. Therefore, the K-matrix method is 
suited to the study of broad and overlapping resonances in multi-channel decays, solving the main 
limitation of the Breit-Wigner model to parameterize the tttt S-wave states in Kg7r~-K~^ [14], 

and avoiding the need to introduce the two a scalars. 

The Dalitz amplitude ^£)(m^, m^) is written in this case as a sum of two-body decay matrix 
elements for the spin-1, spin-2 and Kn spin-0 resonances (as in the Breit-Wigner model), and the 
TTTT spin-0 piece denoted as Fi is written in terms of the K-matrix. We have 

^£)(m?.,m+) = Fi{s) + Z'r^7r7^S-waveare"^''A("^?.,m+) , (3) 

where Fi{s) is the contribution of tttt S-wave states, 

F^{s) = ^[I-iKis)pis)]-;P,{s). (4) 
j 

Here, s is the squared mass of the tttt system (m^+^_), / is the identity matrix, K is the matrix 
describing the /S-wave scattering process, p is the phase-space matrix, and P is the initial production 
vector [11], 

^J\^) - ^2 _ ^'^ Jij „ _ „scatt • 
a ''''a * * *0 

The index j represents the j^^ channel (1 = tttt, 2 = KK, 3 =multi-meson^ , 4 = ryry, 5 = r/r]' [15]). 
The K-matrix parameters are obtained from Ref. [15] from a global fit of the available tttt scattering 
data from threshold up to 1900MeV/c^. The K-matrix parameterization is 

K- (S) - fv ^-"^^ , (l-^^0) (, ,^^2 .ox 



Multi-meson channel refers to a final state with four pions. 
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Table 1: K-matrix parameters as obtained from Refs. [15, 18]. Pole masses (ma) and coupling 
constants (gf) are given in GeV/c^, Sq*^^** and sao are in GeV^/c^. 



rria 






547r 




9rm' 


0.651 


0.229 


-0.554 


0.000 


-0.399 


-0.346 


1.204 


0.941 


0.551 


0.000 


0.391 


0.315 


1.558 


0.369 


0.239 


0.556 


0.183 


0.187 


1.210 


0.337 


0.409 


0.857 


0.199 


-0.010 


1.822 


0.182 


-0.176 


-0.797 


-0.004 


0.224 




i-scatt 
111 


i-scatt 
/l2 


i-scatt 
/l3 


i^scatt 
/l4 


f scatt 
/l5 




0.234 


0.150 


-0.206 


0.328 


0.354 




„scatt 


SAO 


SA 








-3.926 


-0.15 


1 







where gf is the coupling constant of the K-matrix pole to the i^^ channel. The parameters 
yscatt g^j^j gscatt fjggCTibg ^J^e slowly- Varying part of the K-matrix clement. The Adler zero factor 
{l — SAo){s — SAfn'i/'^)/{s — SAo) [16] suppresses false kinematical singularity at s = in the physical 
region near the tttt threshold [17]. Note that the production vector has the same poles as the K- 
matrix, otherwise the Fi vector would vanish (diverge) at the K-matrix (P-vector) poles. The 
parameter values used in this analysis are listed in Table 1 [18]. The parameters ffj^^^, for i ^ 1, 
are all set to zero since they are not related to the vrvr scattering process. 
The phase space matrix is diagonal, Pij{s) = 5ijpi{s), where 



Pi{s) = y 1 , (7) 

with rriii (m2i) denoting the mass of the first (second) final state particle of the i*'* channel. The 
normalization is such that ^ 1 as s ^ oo. We use an analytic continuation of the pi functions 
below threshold. The expression of the multi-meson state phase space is written as [15] 

.X j P31 s<l GeV^ 1 , . 

''^'^ =\PS2 s>l GeV^ I ' 

where 



P3i{s) = po 



dsi ds2 M^T{si)T{s2)^{s + si - S2f - 4gsi 

TT Vr s[(M2-Si)2 + M2r2(si)][(M2 -S2)2 + M2r2(s2) 



P3.(.) = (9) 
s 

Here si and S2 are the squared invariant mass of the two dipion systems, M is the p meson 
mass, and V{s) is the energy-dependent width. The factor po provides the continuity of P3(s) at 
s = 1 GeV^. Energy conservation in the dipion system must be satisfied when calculating the 
integral. This complicated expression reveals the fact that the p meson has an intrinsic width. If 
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Figure 2: (a) The /C°7r"7r+ Dalitz distribution from ^)*^ 

on (b) m^, (c) m^, and (d) m^+^_. Kgir^n^ from D*^ 

The curves are the tttt S-wave K-matrix model fit projections. 



D^TT^ events, and projections 
D^TT^ events are also included. 



one sets r(s) = 6{s), where 6 is the Dirac 6 function, the usual two-body phase space factor is 
obtained. 

Table 2 summarizes the values of the P-vector free parameters Pa and ffl"'^ (we are describing 
only TTTT channel), together with the spin-1, spin-2, and Kn spin-0 amplitudes as in the Breit-Wigner 
model. The third and fifth poles are not included since they are far beyond our tttt kinematic range. 
Figures 2(b,c,d) show the fit projections overlaid with the data distributions. There is no overall 
improvement in the two-dimensional test compared to the Breit-Wigner model [5] since it is 
dominated by the P-wave components, which are identical between the two models. The total 
fit fraction is slightly changed from 1.24 to 1.16. Nevertheless, it should be emphasized that the 
main advantage of using a K-matrix parameterization rather than a sum of two-body amplitudes 
to describe the vrvr S-wave is that it provides a more adequate description of the complex dynamics 
in the presence of overlapping and many channel resonances. 
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Table 2: Complex amplitudes a^e^'^'' and fit fractions of the different components (i^°7r~ and K^i^'^ 
resonances, and 7r"'"7r~ poles) obtained from the fit of the K^t:~t:'^ Dalitz distribution from 

D*+ — > D^'K^ events. Errors are statistical only. Masses and widths of all resonances are taken 
from [2], while the pole masses and scattering data are from [15, 18]. The fit fraction is defined 
for the resonance terms (tttt S-wave term) as the integral of a^|.4r(m^, m^)p (|Fi(s)p) over the 
Dalitz plane divided by the integral of |.4.£)(mi, m^)p. The sum of fit fractions is 1.16. 



Component 


Reja^e*'^'-} 


Imja^e*'^'-} Fit fraction (%) 


if*(892)- 


-1.159 ±0.022 


1.361 ± 0.020 


58.9 


if^(1430)- 


2.482 ± 0.075 


-0.653 ± 0.073 


9.1 


ir|(1430)- 


0.852 ± 0.042 


-0.729 ± 0.051 


3.1 


K*(1410)- 


-0.402 ± 0.076 


0.050 ± 0.072 


0.2 


if*(1680)- 


-1.00 ±0.29 


1.69 ± 0.28 


1.4 


K*(892)+ 


0.133 ± 0.008 


-0.132 ±0.007 


0.7 


K^(1430)+ 


0.375 ± 0.060 


-0.143 ±0.066 


0.2 


K|(1430)+ 


0.088 ± 0.037 


-0.057 ± 0.038 


0.0 


p(770) 


1 (fixed) 


(fixed) 


22.3 


u;(782) 


-0.0182 ± 0.0019 


0.0367 ±0.0014 


0.6 


/2(1270) 


0.787 ± 0.039 


-0.397 ± 0.049 


2.7 


/9(1450) 


0.405 ± 0.079 


-0.458 ±0.116 


0.3 


Pi 


-3.78 ±0.13 


1.23 ±0.16 




P2 


9.55 ± 0.20 


3.43 ± 0.40 






12.97 ±0.67 


1.27 ±0.66 




(.prod 

hi 


-10.22 ±0.32 


-6.35 ± 0.39 




sum of TT+TT" S-wave 






16.2 



5 CP ANALYSIS 

We perform an unbinned extended maximum-likelihood fit to the B~ — > D^K*~ sample to extract 
the CP- violating parameters Xgzf, Vs^, and along with the signal and background yields. The 
Xs± and ys± variables are more suitable fit parameters than ^r^, Sg and 7 because they are better 
behaved near the origin, especially in low-statistics samples. The fit uses mss and the same Fisher 
discriminant as used in [5] to distinguish events from BB production and continuum background. 
The Fisher is a linear combination of four topological variables: Lq = Z^jP*, -^2 = Y^iPW cos0*|^, 
and the absolute values of the cosine of the CM polar angles of the B candidate momcntTim 
and thrust direction. Here, pi and 9* are the CM momentum and the angle with respect to the 
B candidate thrust axis of the remaining tracks and clusters in the event. The likelihood for 
candidate j is obtained by summing the product of the event yield Nc, the probability density 
functions (PDF's) for the kinematic and event shape variables "Pc, and the Dalitz distributions 
-pDaiitz^ over the signal and background components c. The overall likelihood function is 

C = exp I- E ^^c) n E ^cPc(0)7'°^'*^(%) , 

\ c J j c 
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where = {'mES,J^}j, Vj = {rn?L-,'rn\)j, and VdC) = 'Pc{fn^^)'Pc{^)- The components in the 
fit axe signal, continuum background, and BB background. For signal events, 7'^^''*^(r?) is given 
by Tz^{if) corrected by the efficiency variations, where T^{ff) is given by Eq. (2). The mss 
distribution for signal events is described by a Gaussian (double-Gaussian) function distribution 
whose parameters are determined from a fit to the same B~ — > D^Tr~ high-statistics control sample 
as in our previous analysis [5]. 

5.1 Background composition 

The event yields Nc for signal, continuum, and BB components are, respectively, 42 it 8, 251 it 24, 
and 45 it 21, in agreement with our expectation from simulation and measured branching ratios. 
The dominant background contribution is from the random combination of a real or fake 
meson with a charged track and a Kg in continuum events or other BB decays. The continuum 
background in the mES distribution is described by a threshold function [19] whose free parameter ( 
is determined from the B~ D^tt~ control sample. The Fisher PDF for continuum background is 
determined using B~ —>■ D^Tr~ events from the mss sideband region. The shape of the background 
mES distribution in generic BB decays is taken from simulation and uses a threshold function to 
describe the combinatorial component plus a Gaussian distribution to parameterize the peaking 
contribution arising from events with a misreconstructed pion and having a topology similar to 
that of signal events. The Gaussian component has a width of 3.8 ±1.4 GeV/c^ and its fraction 
with respect to the total BB background is 0.13 ± 0.03. The Fisher PDF and the mean of the rriES 
Gaussian for BB events are assumed to be the same as that for the signal. 

An important class of background events arises from continuum and BB background events 
where a real is produced back-to-back with a K* in the CM. Depending on the flavor-charge 
correlation this background can mimic either the 6 ^ c or the 6 — > « signal component. In 
the likelihood function we take this effect into account with two parameters, the fraction f^o of 
background events with a real and the parameter R, the fraction of background events with a 
real associated with an oppositely flavored kaon (same charge correlation as the h u signal 
component). These fractions have been evaluated separately from continuum and generic BB 
simulated events, and are found to be f% = 0.21 ± 0.02, f^f = 0.18 ± 0.02, Ri^ = 0.51 ± 0.06 

and R^^ = 0.67 it 0.06. In addition, to check the reliability of these estimates, the fraction fj^o for 
all background (continuum and BB) events has been evaluated from data using events satisfying 
mES < 5.272 GcV/c2 after removing the requirement on the D mass cut. The measured value of 
0.20 ±0.06 is consistent with the continuum and BB fractions obtained from simulated events. The 
shapes of the Dalitz plot distributions are parameterized by a third-order polynomial in (m?_,m^) 
for the combinatorial component (fake D^) and as signal or shapes for real neutral D mesons. 
The parameters of the polynomials are extracted from off-resonance data and sidebands for the 
continuum component, while we use the simulation for BB. 

A potentially dangerous background originates from signal B~ D'^K*~ where the meson 
is combined with a random K* or pion from the other B meson having the opposite charge. Using 
simulated events the fraction of these wrong sign signal events is found to be (0.43 ± 0.05)%, and 
therefore this contribution has been neglected. 

5.2 CP parameters 

The Dalitz plot distributions for the D'^ ifg7r~7r+ decay from B^ D^K*~ for events with 
mES > 5.272 GeV/c^ are shown in Fig. 3. The distributions for B~ and S+ candidates are shown 
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Table 3: CP- violating parameters {xs±, ys±) obtained from the CP fit to the D^K*~ sample. 

The first error is statistical, the second is the experimental systematic uncertainty and the third 
reflects the Dalitz model uncertainty. 



CP parameter 

Xs- = urs cos{5s - 7) 
ys- = KTs sin((5s - 7) 

Xs+ = KTs COs{6s + 7) 

ys+ = KVs sin((5g + 7) 



Result 

-0.20 ±0.20 ±0.11 ±0.03 
0.26 ± 0.30 ± 0.16 ± 0.03 
-0.07 ±0.23 ±0.13 ±0.03 
-0.01 ±0.32 ±0.18 ±0.05 



separately. The results for the CP-violating parameters Xs± and ys± obtained by fitting those 
distributions are summarized in Table 3. From the same fit we obtain for the value 0.05 ± 0.11 
(statistical error only). The only relevant statistical correlations involving the CP parameters are 
for the pairs {xs-,ys-) and {xs^,ys+), which amount to —10.1% and 2.2%, respectively. The 
statistical correlation of with Xs-, ys-, Xs+, and ys+ are —22.6%, 3.7%, —25.1%, and 22.0%, 
respectively. The Dalitz distribution projections on m^, and m^+^_ for events satisfying 
ruES > 5.272 GeV/c^ are compared to the projection of the fit in Fig. 4, separately for B~ and P+ 
events. Figure 5 shows the two-dimensional one- (dark) and two- (light) standard deviation regions 
(statistical only) in the {xs,ys) plane, corresponding to 39.3% and 86.5% probability content, 
separately for B~ — > D^K*~ and B~^ — > D^K*~^. The separation d between the B~ and B~^ 
regions in these planes is proportional to the amount of direct CP violation, d = 2Krs \ sin 7]. 




12 3 12^ 

m? (GeV^/c^) m? (GeV^/c^) 



Figure 3: The D° K'^tt-tt+ Dahtz distributions from (a) B D^K*- and (b) B+ D^K*+ 
events with ruES > 5.272 GeV/c^. 
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Figure 4: Projections of the Dalitz distributions on m?_, and from a)b)c) D^K*~ 

and d)e)f) D^K*^ events with rn-Es > 5.272 GeV/c^. The projections of the fit result are 

superimposed. 



5.3 Experimental systematic errors 

Table 4 summarizes the break down of the experimental systematic uncertainties. These include 
the errors on the ttt-es and PDF parameters for signal and background, the uncertainties in 
the knowledge of the Dalitz distribution of background events, the efficiency variations across the 
Dalitz plane, and the uncertainty in the fraction of events with a real produced in a back- 
to-back configuration with a negatively-charged kaon. Less significant systematic uncertainties 
originate from the imprecise knowledge of the fraction of real D^'s, the invariant mass resolution 
(negligible), tracking efficiency, and the statistical errors in the Dalitz amplitudes and phases from 
the fit to the tagged sample. We quote as systematic uncertainty, for each effect, the maximum 
of the difference between the bias and square root of the quadratic difference of the statistical error 
between the nominal fit result and the one corresponding to the effect under consideration. These 
systematic uncertainties will be reduced with a larger data and simulated samples and are not 
expected to limit the eventual sensitivity of the analysis. 
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-0.5 0.5 

Figure 5: Two-dimensional one- (dark) and two- (light) standard deviation regions (statistical 
only) in the {xs,ys) plane, corresponding to 39.3% and 86.5% probability content, separately for 
B- D'^K*- (thick and soUd) and B+ D°K*+ (thin and dotted). The confidence regions for 
are superimposed over those for B^ . 



5.4 D° —>■ Kgiv 7r+ Dalitz model systematic uncertainty 

The largest single contribution to the systematic uncertainties in the CP parameters comes from 
the choice of the Dalitz model used to describe the — > i^rg7r~7r+ decay amplitude Ad- We 
use the same procedure as in our previous measurement [5] to evaluate this uncertainty. We first 
generate large samples of pseudo-experiments using the nominal (Breit-Wigner) model. Since the 
effect on the CP parameters depends on their generated values, for each pseudo-experiment we 
randomly generate the truth values of the Cartesian CP parameters according to their measured 
central values and statistical errors. We then compare experiment by experiment the values of 
the Xs± and ys± obtained from fits using the nominal model and a set of alternative models. We 
find that models resulting by removing different combinations of higher K* and p resonances (with 
low fit fractions), or changing the functional form of the resonance shapes, has little effect on the 
total of the fit, or on the values of the CP parameters (at most 1° for 7 and 0.002 for Kr^). 
As an extreme we consider a model without the ai and/or a2 scalar resonances, or the CLEO 
Breit-Wigner model [9]. Fits to these models result in a significantly larger than that of the 
nominal model, but the effect on the CP parameters are still small, as indicated in Table 4. These 

uncertainties translate to Kr^, 7 and 6s as 0.05, 9°, and 11°, respectively ^. For B~ — > D^*^^K~ 

(*) (*) 

decays, the same procedure to propagate the uncertainties on , 7, and 5^ gives 0.027(0.027), 
11°, and 14°(13°), respectively. 

As an additional cross-check of the fact that models without the a scalars are in fact an extreme 

®The Dalitz model uncertainties on nVs, 7 and Ss are obtained by repeating the fits to the same high statistics 
pseudo-experiments but now fitting directly to these CP parameters. This method accounts for the systematic 
correlations among the Cartesian CP-violating parameters. 
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Table 4: Summary of the non-negligible contributions to the systematic error on the CP parameters 
Xs± and ys±. 



Source 




Vs- 


Xs+ 


ys+ 


mES) ^ shapes 


0.08 


0.12 


0.10 


0.12 


Background Dalitz shape 


0.04 


0.09 


0.04 


0.09 


Efficiency in the Dalitz plot 


0.06 


0.04 


0.07 


0.09 


Right sign f»° fractions R^^) 


0.03 


0.04 


0.03 


0.05 


Real £»° fractions (/^^o, fEh 


0.03 


0.03 


0.03 


0.04 


Tracking efficiency 


0.01 


0.01 


0.01 


0.01 


Dalitz amplitudes and phases 


0.01 


0.01 


0.01 


0.01 


Total experimental 


0.11 


0.16 


0.13 


0.18 


Dalilz model (Breil-AMgner un)del williout a sealars) 


0.03 


0.03 


0.03 


0.05 


Dalitz model (tttt S-wave K-matrix model) 


0.01 


0.01 


0.01 


0.01 



case, we repeated the above procedure using as alternative model the tttt S-wave K-matrix model 
described in Sec. 4 instead of the CLEO Breit-Wigner model (the latter used to quote the Dalitz 
model systematic uncertainty, as described before). The effect on the Cartesian CP parameters is 
more than three times smaller, as shown in Table 4. These uncertainties translate to Kr^, 7 and 
5s as 0.015, 3°, and 2°, respectively. For B~ — > D^*'>^K~ decays, the scaling of the effect on the 
Cartesian CP parameters is similar to B~ DK*~ . In terms of , 7, and 5^ the variation is 
0.004(0.003), 2°, and 2° (2°), respectively. 

6 INTERPRETATION AND RESULTS 

A frequentist (Neyman) analysis [2] has been adopted to interpret the constraints on z± = {xs±,ys±) 
in terms of p = {kTs^Ss^i)- We construct an analytical parameterization of the four-dimensional 
probability density function V of z± as a function of p, 

(z+,z_|p) = -2 — (z+ip)-2 — (z_|p) , (10) 



where 



(z±|p) = G2[7-±;KrsCos{5s±-t),KrsSui{6s±-t),axs±,cFy^±,Ps±] (H) 



is a two-dimensional Gaussian distribution [20]. Here, ps± represents the correlation between Xs± 
and ys±- For a given p, the three-dimensional confidence level is estimated as C(p) = 1 — a(p), 
where q(p) is calculated by integrating Eq. (10) over a domain defined by all points in the four- 
dimensional fit parameter space closer (larger PDF) to p than the fitted data values, 

d^V d^V 

(z+,z_|p)>-5-4-(zf%zi-^^|p). (12) 



(i2z+(i2z_ ' d2z+d2z_ 

The one (two) standard deviation region of the CP parameters is defined as the set of p values 
(constructed from a large number of pseudo-experiments) for which Q;(p) is smaller than 19.88% 
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Figure 6: Two-dimensional projection onto the KVg — 7 plane of the three-dimensional one- (dark) 
and two- (light) standard deviation regions, including statistical and experimental systematic un- 
certainties, for B~ D^K*~. 



(73.85%). Figure 6 shows the constraints in the — 7 plane as obtained by projecting the three- 
dimensional confidence regions, including statistical and experimental systematic uncertainties. 

B- D^K*- events can be used in combination with B — > D^*^'^K [5] to improve the 
overall constraints on 7. The procedure used to combine the three B decay channels is identical 
to the one described above, but with increased number of dimensions. In this case, the dimension 
of the fit parameter space is twelve, z± = {x± ,Xs±,ys±)- The CP parameter space has 
instead seven dimensions, p = {r^^\6^^\ Krs,6s,"f)- The one (two) standard deviation region of 
the CP parameters is defined as the set of p values for which a(p) is larger than 0.52% (22.02%). 
Figure 7 shows the two-dimensional projections onto the r^— 7, r^j— 7, and Kr^— 7 planes, including 
statistical and experimental systematic uncertainties. The figures show that this Dalitz analysis has 
a two-fold ambiguity. The combination if the three signal modes yields 7 = (67 it 28 it 13 it 11)°, 
where the first error is statistical, the second is the experimental systematic uncertainty and the 
third reflects the Dalitz model uncertainty. Of the two possible solutions we choose the one with 
< 7 < 180°. The contribution to the Dalitz model uncertainty due to the description of the tttt 
b-wave m vr vr"*" is 3°. From this combination, Kr^ is constrained to be < 0.50 (0.75) at 

one (two) standard deviation level. It is worth noting that the value of nr^ depends on the selected 
phase space region of B^ D^(K^tt~) events without introducing any bias on the extraction of 7. 

The constraint on 7 is consistent with that reported by the Belle Collaboration [6, 7]. However, 
the statistical error turns out to be larger than that of Belle because our data favors smaller values 
of and Atr^ . Simulation studies confirm that the difference in statistical errors is consistent with 
the scaling expected from the different actual values. 
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Figure 7: Two-dimensional projections onto the (a) rs — 7, (b) — 7, and (c) Kr^ — 7 planes of 
the seven-dimensional one- (dark) and two- (light) standard deviation regions, for the combination 
of B- and B~ D^K*' modes. 
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